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THEORY  AND  APPLICATION  OF  THE  TRUNCATED  NORMAL 
DISTRIBUTION  FOR  REMOTELY  SENSED  DATA1 

David  S  Crosby- 

National  Oceanic  and  Atmospheric  Administration 
The  American  University 

D.  J.  DePriest 
Office  of  Naval  Research 

I.  INTRODUCTION 

A  fundamental  problem  in  measuring  the  infrared 
radiation  emitted  from  the  surface  or  lower  atmosphere 
of  the  earth  is  the  presence  of  clouds.  The  usual  effect 
of  clouds  in  an  instrument  field-of-view  is  to  lower  the 
measured  radirtion.  In  order  to  be  able  to  use  many  of 
the  techniques  of  remote  sensing  it  is  necessary  to  have 
an  estimate  of  the  emitted  radiance  from  a  clear  or  cloud- 
free  field-of-view.  The  technique  which  is  developed  in 
this  paper  uses  the  measured  radiances  from  a  large  number 
of  neighboring  small  fields-of-view.  If  a  sufficient 
number  of  these  small  fields-of-view  are  cloud-free,  then 
a  technique  which  uses  the  theory  of  the  truncated  normal 
distribution  can  be  applied  to  the  data  to  obtain  an  esti- 
mat  the  clear  radiance  (Crosby  and  Glasser,  1978). 

1Part  of  this  paper  was  written  while  Dr.  Crosby  was 
supported  by  the  Office  of  Naval  Research  Contract  N00014- 
77C-0624. 


Since  the  technique  developed  is  a  maximum  likelihood  pro¬ 
cedure,  standard  statistical  procedures  can  be  used  to 
determine  if  the  distribution  in  the  tail  is  approximately 
normal.  This  allows  for  a  simple  decision  procedure  to 
check  if  the  model  is  satisfied  (DePriest  1976,  Crosby  and 
DePriest  1977).  These  decision  procedures  use  classical 
statistical  goodness-of-fit  tests.  Finally,  one  difficulty 
with  the  application  of  this  procedure  has  been  an  objective 
method  of  determining  a  truncation  point.  A  sequential 
procedure  for  determining  this  truncation  point  is  outlined. 

II.  THE  MODEL 

In  order  to  apply  the  technique  of  the  truncated  normal 
the  following  conditions  must  be  satisfied.  (1)  There  are 
a  number  of  neighboring  measurements  where  the  radiances  in 
the  absence  of  clouds  would  be  the  ;'ame.  (2)  The  presence 
of  clouds  lowers  the  radiance  in  the  field-of-view .  (3)  Some 

of  the  f ields-of-view  are  cloud-free.  (^)  The  difference 
between  the  measured  radiances  from  two  cloud-free  fields-of- 
view  is  due  only  to  instrument  noise,  whose  distribution  is 
assumed  to  be  normal. 

It  is  assumed  that  the  density  of  the  measured  radi¬ 
ances  from  all  the  fields-of-view  in  a  set  takes  the  form 


-  •  ■'  - 


h(x)  =  «  f(x)  4  (1  -  «)  g(x)  .  ( 

In  equation  1,  d  is  equal  to  the  proportion  of  fields-of- 
view  which  are  cloud-free, 

f(x)  =  ( 

and  g(x)  is  some  unknown  density  function.  It  is  further 
assumed  that  there  is  some  T  such  that 

g(x)  =  0  x  >  T.  ( 

That  is  for  the  values  of  x  above  T,  the  density  curve  has 
the  form  of  a  normal  density  curve  with  parameters  p  and  o' 
The  parameter  p  is  the  true  clear  radiance.  That  is  p  is 
the  radiance  that  would  be  measured  from  a  clear  field-of- 
view  if  there  were  no  instrument  noise.  The  variance,  cT , 
of  the  instrument  noise  is  assumed  to  be  known.  The 
problem  is  to  obtain  a  good  estimate  of  p. 

III.  THE  ESTIMATION  PROCEDURE 

The  statistical  technique  outlined  in  this  section  is 
well  known  and  is  available  in  the  literature.  See,  for 
example,  Crosby  and  Glasser  197?. 

If  the  radiance  data  above  some  truncation  point  T 
is  from  a  truncated  normal  distribution,  that  is  it.  has 
the  form  given  by  equation  2,  and  if  it  has  known  variance 
then  the  maximum  likelihood  estimate  for  the  mean  p 
is  a  monotonic  function  of  the  arithmetic  mean  of  the 
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observations  above  that  truncation  point. 

Let  IX10  "  1>  •••,  n)  be  the  measured  radiance  values 


above  some  truncation  point  T.  Let 

Zi  "  ' 

T)  /er 

(*) 

and  let 

v  =  (T  -  |i) 

/r  . 

(.) 

Recall  that  T  is  a  known  truncation  point,  a  is  the  known 
instrument  noise,  and  u  is  the  estimate  of  the  clear 
radiance.  Let 


n 

2  -  I  Z1  (6) 

1=1 

then  v  is  given  by  the  solution  to  the  equation 

co 

-v  +  (exp(-v2/2)  /  f  exp(-xc'/2)dx)  -  7  .  (7) 

'  v 


Once  v  is  known,  u  is  found  by  the  inverse  of  equation  ty-'' 

A  m  A 

VI  -  T  -  v  C  . 

Since  the  function  given  in  equation  8  is  monotonic  and 
smooth,  so  is  its  inverse.  It  is  possible  to  find  a  good 


rational  polynomial  approximation  to  the  inverse  function 
which  gives  v  as  a  function  of 
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As  an  example  this  procedure  has  been  applied  to  the 
10.5  -  12.5  Uni  infrared  channel  of  the  Scanning  Radiometer 
of  the  NOAA  satellites.  Table  1  gives  the  upper  tail  of 
one  such  data  set.  The  measurements  are  in  counts  and 
have  been  transformed  in  order  to  keep  the  example  as 
simple  as  possible. 


TABLE  1 

Data  from  the  10.5  -  12.5  um  infrared  channel  of  the 
Scanning  Radiometer. 


Counts 

Frequency 

Estimated  Mean 

C 

fl 

a 

u 

73 

c : 

72 

3 

71 

6 

70 

12 

69 

22 

66 

31 

65.0 

For  the  data  in  Table  1,  T  =  67.5,  c?  =  3  and  hence 
7  =  .560.  Then  v  =  .835  and  u  =  67.5  -  ((.833)5)  =  65. 0. 
The  value  for  e r  is  found  from  independent  data  and  v  is 
found  from  7  by  solving  equation  7. 

IV.  G00DNESS-0F-FIT  TESTS 

Ihe  procedures  outlined  in  section  III  depend  on  the 
assumption  that  the  upper  tail  of  the  distribution  has  the 


shape  of  a  truncated  normal  distribution.  If  the  shape  is 
not  normal  it  may  indicate  that  few,  if  any,  of  the  fields- 
of-view  are  completely  cloud-free.  Hence  the  shape  of  the 
data  from  the  tail  may  be  used  to  screen  out  situations  of 
this  type. 

The  test  which  is  recommended  is  the  chi-square 
goodness-of-fit  test.  For  a  description  of  other  possible 
tests  see  DePriest  1976.  The  chi-square  test  is  well 
known.  See,  for  example,  Cochran  1952.  However,  for  com¬ 
pleteness,  a  short  description  of  the  procedure  will  be 
given  here. 

Given  n  observations  grouped  into  K  mutually  exclu¬ 
sive  categories  let  0i  and  E^  (i  =1,  2,  .  ••,  k)  denote 
respectively  the  observed  and  expected  frequencies  for 
the  K  categories.  The  statistic  is 

X?  =  £  (0i  -  E  )2/E  .  (9) 

i=l  iii 

A 

Since  the  estimate  of  the  parameter  p,  y,  is  a  maximum 
likelihood  estimate  the  asymptotic  distribution  of  the 
statistic  Xc  will  be  chi-square  with  K  -  2  degrees  of 
freedom.  Since  the  radiance  values  are  already  grouped, 
this  natural  grouping  is  used  in  the  computation  of  the 
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statistic.  To  illustrate  the  technique,  two  examples  are 
provided . 

For  the  data  in  Tables  2  and  3  it  was  assumed  that 
CT  »  3 ,  This  value  was  determined  from  independent  measure¬ 
ments. 

TABLE  2 

Data  From  the  Scanning  Radiometer 

The  data  are  in  counts.  The  estimated  mean  uses  the  data  i 
the  tail  above  and  including  the  corresponding  count. 

Counts  Frequency  Estimated  Mean 


c 

IT 

Ji 

7^ 

| 

C. 

7? 

71 

8 

70 

12 

69 

22 

68 

31 

o7 

42 

66 

34 

69 

37 

64 

40 

For  the  data  in  Table  2  a  truncation  point  of  6^.9  was 

A 

take*  This  gives  an  estimate  of  p,  u  -  0^.3.  The  number 

of 


'ps  was  taken  to  be  equal  to  9.  The  data  associated 


with  72  and  77  counts  are  grouped  into  a  single  class.  The: 

the  chi-square  statistic  will  have  7  decrees  of  freedom.  It 

<•> 

is  found  that  X‘  »  This  value  indicates  a  good  fit. 

The  model  given  in  section  II  is  reasonable. 


TABLE  7 

Data  From  the  Scanning  Radiometer 

The  data  are  in  counts.  Tito  estimated  mean  uses  the  data 
in  the  tail  above  and  including  the  corresponding  count. 

Counts  Frequency  Estimated  Mean 

A 

fi  »* 

o 

»■* 

( 

A 

V| 

6 

Q 

v. 

6 

For  the  data  in  Table  a  truncation  point  of  was 

taken  and  (j  =  Grouping  the  counts  ?0  and  fco  into 

one  class,  there  are  o  classes.  The  chi-square  statistic 
has  h  degrees  of  freedom.  Tito  value  is  found  to  be  1.'.'. 
The  fit  is  not  very  good  and  the  model  of  an  upper  tail 
which  is  normal  is  rejected. 


V.  DETERMINING  THE  TRUNCATION  POINT 


Giver,  a  truncation  point  T,  the  procedure  outlined  in 
the  earlier  sections  will  estimate  u  ar.u  test  if  the  tail 
is  normal.  However  one  difficulty  with  the  application  of 
these  techniques  has  been  a  determination  cf  a  good  trun¬ 
cation  point.  The  following  procedure  is  recommended.  It 
is  assumed  that  the  data  is  grouped .  Then  the  count  data 
from  the  instrument  will  be  in  the  format  represented  by 
Table  4. 

TA3LE  4 

Class  Class  Midpoint  Class  Frequency 

*0-*!  m1  =  (kj+kQj/2  f1 

ki-kg  =  (kg+kjJ/2  f2 


ki-l-ki  mi  =  (-‘i+k^_j  )/2 

The  minimum  number  of  classes  used  in  the  technique  should 
cover  approximately  2  standard  deviations.  In  the  examples 
which  have  been  presented  er  was  equal  to  3,  and  a  minimum 
of  7  classes  was  used.  The  adjusted  mean  of  the  data  above 
and  including  class  j  is 


A 

Then  v is  found  using  equation  7  or  an  approximation  to 

c 

A  - 

this  equation.  If  v.  >  0,  compute  a  Z.  , .  Repeat  this 
process  until  v  <  0.  The  data  above  the  point  k^  is  then 
checked  using  the  goocness-of-f it  criteria  of  section  4. 

If  the  data  does  not  fit  the  tail  of  a  normal  distribution 
the  process  is  terminated  and  the  retrieval  discarded. 

At  this  stage  of  the  process  the  truncation  point  is 
1c  .  The  truncation  point  should  be  selected  to  include 
as  much  of  the  data  as  possible.  The  reason  for  this  is 
seen  in  Table  5.  Table  5  gives  the  asymptotic  standard 
deviation  of  ((£  -  ^Tfor  various  values  of  T.  In 

this  table  N  is  equal  to  the  total  sample  sine  if  there 
were  no  truncation,  a  is  the  known  standard  deviation,  and 
p  is  the  estimate  of  p  found  using  the  maximum  likelihood 
technique  of  section  3.  It  is  seen  from  this  table  that 
there  is  a  significant  reduction  in  the  standard  error  of 
estimate  for  p  i f  the  truncation  point  changes  from  p  to 
two  standard  deviations  below  p. 
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TABLE  ^ 


Asymptotic  Standard  Deviation  of  ((u  -  u)%^)/<r 


Asymptotic  Standard  Deviation 

3.474 

2.346 

1.373 

1.17? 

1.074 

1.007 


(In  Table  S  T  =*  (T  -  p) At. ) 


A  sequential  procedure  is  used  to  determine  if  the 
truncation  point  should  be  moved.  Let  IXo  *  IXrQ 

be  the  radiance  measurements  above  some  Tj  and  let 
IA1,  I^<;,  •••,  IAnl,  •••,  IXn-  be  the  radiance  measure¬ 
ments  above  some  To(To  <  T^ )  .  A  decision  must  be  made 
whether  to  move  the  truncation  point  to  To  and  hence  increase 
the  number  of  measurements  used  to  estimate  u.  The  decision 
process  is  based  upon  the  result  that  usih£  the  information 
from  the  values  1^,  Ix  ,  •••,  IXrj(  it  is  possible  to 

estimate  n„,  This  estimate  is  called  n„.  It  can  also  be 

*  i  c 

A 

shown  that  (no  -  no)/ho  is  asymptotical  normal  with  mean 

•  «-  *-  JL 

0  and  cstimatable  standard  deviation.  If  (nA  -  n.,)  is 

t.  *  t 

within  certain  bounds  then  the  truncation  point  is  extended 

a  £ 

to  To.  If  (rio  -  n«)/n0  is  outside  these  bounds,  then 

i.  '  t  L.  '  C 


wherr  [y]  denotes  the  preatest  integer  less  thou  or  equal 
to  y,  Lot  ru  be  the  actual  number  of  measuroments  above 
T,,.  Assuming,  that  1^,,  >  come  from  a  popu¬ 

lation  which  Is  normal  with  mean  |i  and  variance  r‘  -  1, 
the  asymptotic  variance  of  (ft,  -  n,)  n  is  pi  von  by 

«  4. 


1.' 


2  =  (Fc(T2'M)V  (T1,M)-Fc(TI,u)Vi>(T2>M))1 
fc(T2,M)  (FjT^p))  HfT^p) 

fc(T2»M)  -  ^(T^fi) 


Fc(TrM) 


(13) 


where 


V^(T,p)  =  (1/2*)“*  exp  - ( T  - 

p)2/2 

(14) 

. 

1 

ti 

Fc(T,p)  =  (1/2tt)-  * 

9T 

J 

T 

exp-(x  -  v*)2/2dx 

(13) 

£ 

H(T1,p)  =  F^T^p) 

S3 

1 

i — i 

E-i 

+ 

)  •  fc(t1>m)H'(t1,h) 

-h,(t1  ,nr  . 

Table  -  gives  the  asymptotic  standard  deviations  for  some 

values  of  T-^  and  T0  when  p 

-  0. 

TABLE  6 

• 

! 

The  Asymptotic  Standard 

Deviations  of  (n0  -  n^)  'n„* 

c.  c.  '  c. 

1 

T1 

T2 

s 

1.0 

.67 

1.0s  4 

0.5 

.17 

.833 

0.0 

-.53 

.032 

L 

-0.5 

-.83 

.  4o0 

-1.0 

-1.33 

.316 

-1.5 

-1.8? 

.202 

-2.0 

-2.33 

.119 

-2.5 

-2.83 

.064 

. 
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• 

J 


<> 


This  sequential  procedure  has  been  applied  to  data 
from  the  10.5  -  12.5  pm  channel  of  the  Scanning  Radiometer. 
Table  7  gives  the  upper  tail  of  such  a  data  set.  The  data 


are  in  counts . 


TABLE  7 

Data  From  the  Scanning  Radiometer 


Counts 

Frequency 

Estimated  Mean 

(n2-n2)/n2‘ 

S 

86 

1 

67 

5 

86 

11 

65 

18 

76.5 

84 

51 

79.0 

65 

47 

79.5 

82 

67 

79.4 

Si 

68 

79.7 

80 

71 

79.9 * 

. f 

.58 

79 

75 

79.6 

-.10 

.46 

78 

60 

79.9 

.15 

.57 

77 

57 

79.8 

-.52 

.29 

76 

45 

79.7 

-.40 

.25 

7S 

1  > 

44 

79  •  4 

-.6? 

.18 

74 

28 

79.5 

75 

22 

79.2 

72 

lo 

79.1 

71 

26 

7  6.8 

70 

21 

78.5 

69 

9 

68 

21 

67 

20 

66 

15 

is  less  than  zero. 


For  this  example  c r  =  3.  The  estimate  of  p,  p  uses  the  data 
in  the  tail  above  and  including  the  corresponding  counts. 
For  example  the  estimate  correspond! ng  to  8 0  uses  a  trun¬ 
cation  point  of  7Q.r'.  This  is  the  first  estimate  where  v 
is  less  than  zero.  The  sequential  procedure  is  then  aprlie 
For  the  other  entries  corresponding  to  60  T,  =  79. c,  T-  = 
78.8,  n0  =  31"’ ,  nc  7  3^7  and  n,  =  246.  Note  that 

d.  c.  X 

,A  .  .  1 

( n-  -  n-j/n^2  is  not  large  compared  to  S  and  hence  the 
truncation  point  would  be  moved  to  T^  or  79.6.  Continuing 
this  process,  if  a  conservative  rule  were  used  the  process 
would  terminate  at  T  =  76.5  which  would  yield  an  estimate 

A 

of  p  =  79.7.  3y  any  reasonable  rule  the  process  would 
terminate  at  T  =  74.5  which  gives  a  p  =  79.4.  Note  for 
this  last  entry  the  value  of  the  statistic  (n0  -  n..)/n?^ 
is  greater  than  four  standard  deviations  from  0. 

VI .  CONCLUSIONS 

In  this  paper  a  procedure  which  uses  standard  and  new 
statistical  techniques  to  estimate  clear  radiances  for  a 
high  spacial  resolution,  multiple  field-of-view  infrared 
instrument  has  been  developed.  The  technique  does  not  com¬ 
pletely  solve  the  cloud  problem.  For  example,  it  will  not 
distinguish  between  a  low  level  uniform  cloud  layer  and  the 
surface.  Also,  certain  types  of  inversions  in  the 


temperature  profile  will  cause  difficulty.  However,  the 
technique  is  computationally  simple  and  can  easily  be  used 
with  other  types  of  procedures.  This  technique  has  been 
applied  to  reel  data  and  seems  to  work  very  well. 
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1.  INTRODUCTION 

A  fundamental  problem  in  measuring  the  infrared  radiation 
emitted  from  the  surface  or  lower  atmosphere  of  the  earth  from 
space  is  the  presence  of  clouds.  The  usual  effect  of  clouds 
in  an  instrument  field-of-view  is  to  lower  the  emitted  radi¬ 
ance.  In  order  to  use  many  of  the  techniques  of  remote  sen¬ 
sing  it  is  necessary  to  have  an  estimate  of  the  emitted  radi¬ 
ance  from  a  clear  or  cloud-free  field-of-view.  One  technique 
which  has  been  used  successfully  is  to  measure  the  radiation 
emitted  from  a  large  number  of  neighboring  small  fields-of- 
view.  If  a  sufficient  number  of  these  small  fields-of-view 
are  cloud-free,  then  a  technique  which  uses  the  theory  of 
the  truncated  normal  distribution  can  be  applied  to  the  data 
to  obtain  an  estimate  of  the  clear  radiance  (Crosby  and 
Glasser,  197$).  One  difficulty  with  the  application  of  this 


1 


technique  has  been  determing  an  objective  method  of  selecting 
the  truncation  point.  In  this  paper  a  sequential  procedure 
for  determining  the  truncation  point  is  developed. 

2.  THE  MODEL 

In  order  to  apply  the  technique  of  the  truncated  normal 
the  following  conditions  must  be  satisfied.  (1)  There  are 
a  number  of  neighboring  measurements  where  the  radiances  in 
the  absence  of  clouds  would  be  the  same.  (2)  The  presence 
of  clouds  lowers  the  radiance  in  the  field-of-view.  (3)  Some 
of  the  fields-of-view  are  cloud-free.  (4)  The  difference 
between  the  measured  radiances  from  two  cloud-free  fields-of- 
view  is  due  only  to  instrument  noise,  whose  distribution  is 
assumed  to  be  normal. 

It  is  assumed  that  the  density  of  the  measured  radiances 
from  tv.o  cloud-free  fields-of-view  is  due  only  to  instrument 
noise,  whose  distribution  is  assumed  to  be  normal. 

It  is  assumed  that  the  density  of  the  measured  radiances 
from  all  the  fields-of-view  in  a  set  taxes  the  form 

h(x)  =  C  f(x)  +  (1-flQ  g(x) .  (1) 

In  equation  1,  tfis  equal  to  the  proportion  of  fields-of- 
v-' ev  ich  are  cloud-free. 


f  ( x )  =  (1/2TT02)*  exp(-(x-n)2)/2cr^). 


(2) 


and  g(x)  is  some  unknown  density  function.  It  is  further 
assumed  that  there  is  some  T  such  that 

g(x)  =0  x  >  T.  (?) 

That  is  for  the  values  of  x  above  T,  the  density  curve  has 

2 

the  form  of  a  normal  density  curve  with  parameters  p  and  a  . 
The  parameter  p  is  the  true  clear  radiance.  That  is  p  is 
the  radiance  that  would  be  measured  from  a  clear  field-of- 
view  if  there  were  no  instrument  noise.  The  variance,  O'", 
of  the  instrument  noise  is  assumed  to  be  known.  The  problem 
is  to  obtain  a  good  estimate  of  p.  If  T  is  known  then  stan¬ 
dard  results  can  be  applied  to  obtain  an  estimate  of  p. 
(Crosby  and  Glasser,  197^).  The  point  T  is  in  general 
unknown;  however,  it  is  assumed  there  is  a  known  T^(T^  >  T) 
such  that  g(x)  =  0  for  x  >  T^ .  The  T  in  equation  ?  is  the 
optimal  truncation  point. 

?.  SEQUENTIAL  PROCEDURE  FOR  ESTIMATING  T 

Let  X^,  X^,  •••,  Xn  be  a  random  sample  from  the  popu¬ 
lation  with  the  density  in  equation  (1).  Let  X^,  X^0, 

*  *  * ,  Xln  be  the  values  from  the  sample  which  are  above  the 
known  T^  (T^  is  defined  in  section  2).  A  T?  (T^  <  T^)  is 
selected.  The  values  X-^,  X^,  •••,  Xin  are  used  to 

^  a 

estimate  p.  Let  the  maximum  likelihood  estimator  be  p^.  Let 


00 

FC(T.  ,  A,  0C )  =  J  e\T(-(x-J)/2cr‘- )dx. 


(M 


Given  the  values  X^, 
the  number  of  measurement 
distribution  above  T„  is 


,  ••*,  X^n  and  ju,  ,  an  estimator  of 

a  ^ 

s  n.,  above  T^,  assuming  that  the 


normal,  is  given  by 


*2  -  tni/(Fc(Tl*  »i>  O  /  FC(T2*  $1'  ^))]  (5) 

where  [y]  denotes  the  greatest  integer  less  than  y.  Let  r. 
be  the  actual  number  of  values  above  Tn.  If  the  distribution 
is  normal  above  T,  then  it  is  possible  to  show  that 

C. 


(rL  -  n„)  /  (n,)  ^ 

is  asymptotically  normal,  where  this  normal  has  near.  0  ar.d 
known  standard  deviation. 

If  -  n„)  /  (n.v)  '  is  outside  of  certain  bounds  as 
determined  by  the  asymptotic  theory,  then  is  used  as  T 
and  u.  is  used  as  the  estimator  of  ii.  If  (ft.,  -  n,)  ^  In.)  * 

1  '  t  £  1  /  '  L  1 

is  within  these  bounds,  then  TL,  is  used  as  a  truncation 
point  and  a  1\(T-  <  T0)  is  selected  and  the  process  repeated 
with  replacing  and  replacing  T?.  The  process  is 
repeated  until  the  (nj  -  n^)  J  (n^ )*  is  outside  the  bounds 
or  until  (Ti  -  u)  /  o  is  less  than  some  C, 
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-  .  AS YM P TO TI C  THEO RY 

Itie  results  of  this  section  are  based  er.  the  theory  fiver, 
in  a  series  of  oarers  by  Sanathanan  (197ffl,  1977). 

Let.  X,,  X„,  Xj,  be  N  values  of  a  variable  X  indepenc.- 

ently  sampled  from  a  distribution  F(.\).  Assume  that  two  con¬ 
stants  are  given  and  (T-.  <  T^)  such  that  the  distri¬ 
bution  F(\)  has  -trow:.  functional  form.  F(x;p)  to  the  right  of 
T  .  Let  n-  equal  the  number  of  variables  to  the  right  of  T. . 
In  the  argument  which  follows  n,  is  considered  to  be  a  nara- 
r.eter.  Consider  the  distribution  of  the  variables  above  T  . 
This  is  a  truncated  distribution  with  density  function 


[  rr  . 


where  f(x;  n)  is  the  density  of  F(x;  \\)  and  F  (T-;  p)  = 

1  -  F(T-.;  >i) .  Call  this  distribution  function  FT  (j:''  .  The 
distribution  function  of  the  variables  above  T-,  can  be  con¬ 
sidered  as  a  truncated  distribution  of  the  distribution  F„  (j;''  . 
The  density  function  of  this  distribution  will  have  form 


Fc(Tr:  H) 

FC(T1? 

fc(t2;  fx) 


- 

fTHV 


K 


f^XJ  Jl) 


Note  that 


rc(y  *■) 

fc(T2;  ,.) 


=  P(p)  is  the  conditional  probability 


that 


nl  * 


X.  is  above 
Let  X-, ,  X  , 
n,~,  aoove  T,  . 

C  1 


T^  given  that  it  is  above  T,  . 

• • • ,  X„  be  a  random  number  of  observations 
nl 

The  likelihood  function  is  given  by 


L(n«;  p)  =  ( n2 ;  P(jp) ) 


(V 


where 


L1(n2;  P(p))  = 


n0! 


(P(p))nl(l  -  P(|i)) 


nl •  (n£  -  n1)! 


n^-n, 

a  1 


ana 


MjO  =  *  g(x.  ;  p) 

i-1 


(6) 


(?) 


Only  the  case  where  (n^,  p)  is  maximised  first  by  obtaining 
p  as  the  value  of  p  which  maximized  Lr.  (^)  and  then  maximizing 
L1(n2,  P(fO)  with  respect  to  is  considered.  This  estimate 
of  n?  is  given  by  ft.  =  [n^  /  P[ju]J  where  [y]  denotes  the 
greatest  integer  less  than  y.  Then  as  shown  by  Sanathanan 
(1977)  if  at  every  admissible  value  of  p  (p  is  a  r-dimensi er.al 
vector  of  parameters)  P(p)  admits  continuous  first-order 
partial  derivatives  and  f(x,  n)  admits  continuous  first  and 
second  order  partial  derivatives  almost  everywhere  with 
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respect  to  x  then 


(  7^  ,  u,  P(/»))->(l,  y,  P(|i)) 

C 


a.s . 


and 


,A 

(u 


u); 


is  asymptotically  normal  N(0,  Z)  where  Z”1  is  the  (r  +  1)  x  (r  +  1) 
matrix  given  below. 

Let  the  partial  derivatives  of  P(p)  and  log  g(x,  p)  with 

respect  to  .  (the  j*'*1  component  of  ji)  be  denoted  by  PiOO. 

%)  J 

g  .  (x  ,  p)  .  Then  set 


p.lW  Pm(^) 

P(p)  (1  -  P(|i) ) 


+  P(H)  • 


(10) 


E(g-;(x;  p)  gm(x;  p)). 


(The  expected  values  are  taken  with  respect  to  g(x  ;  i0). 


n  ,  P» 

om  “  TTpT 


a  =  P(p) 

oo  r  ”  p(p) 


The  inverse  of  the  matrix  T.  is  given  by 
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(11) 


(1?) 


.-1 


,1m  ^ 

(■<*) 

i  >m^ 

(a  ) 

'  oo ' 

(1?) 


For  further  discussion  and  development  see  the  papers  by 
Sana  t  hanan . 

If  the  above  theory  Is  applied  to  the  single  parameter 
truncated  normal  with  variance  1,  It  Is  found  that  the 
asymptotic  variance  of  (A,  -  n.,)  n"  Ms  given  l\v 

•  c. 


S* 


4- 


(KC(T.;  )Q  -  FC(T];  /»))V(T,;  ji))’ 

FC(T,;  ;0  (Pc(Tji  >0)  H(Tl4  >0 

Fc(y.  v)  -  Fc ( ;  V) 

( Ti :  *0 


( i ) 


who  *'e 


^(T:  ju)  -  (1/2tt)‘*  exp  -  ( T  -  )>)'’/? 


U") 


FC(T;  fi)  *  (l/2ir)"  *  J  exp  -  (x  -  >0*  .  dx  (  lo) 

T 

H(TlS  ix)  -  fJ(T,;  ix)  4  (Tj  -  *>)  •  )T  ( Tt :  /i)- V  ( Tx  s  >0’' 

(17) 


Table  1  gives  this  asymptotic  variance  for  some  values 
Note  that  in  the  application  )>  is  estimated 

-  P  - 


of  Tj  and  Tj, 


and  hence  so  is  the  asymptotic  variance. 

TABLE  1 


The 

.A 

Asymptotic  Variance  of  (n0 

-  n2)n2"  * 

T1 

T0 

d 

fs 

Sc 

1.0 

0.5 

2.39 

0.5 

0. 

1.33 

0. 

-0.5 

0.700 

0.5 

-1.0 

0.340 

1.0 

-1.5 

0 . 14£: 

1.5 

-2.0 

0.057 

2.0 

-2.5 

0.019 

2.5 

-3.0 

0.005 

5.  EXAMPLE 

The  procedure  has  been  applied  to  the  10.5  -  12.5  urn 
infrared  channel  of  the  Scanning  Radiometer  on  the  NOAA 
satellites.  The  data  appears  in  blocks  of  1024  measure¬ 
ments.  Table  2  gives  the  upper  tail  of  one  such  data  set. 
The  measurements  are  in  counts  and  have  been  transformed  in 
order  to  keep  the  example  as  simple  as  possible. 

For  this  example  it  is  assumed  =  3.  This  value 
was  determined  from  independent  measurements.  It  is  clear 
from  an  examination  of  the  data  that  a  reasonable  trunca¬ 
tion  point  for  this  data  is  57.5. 
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TABLE  2 


Data  From  the  Scanning  Radiometer 

The  data  are  In  counts .  The  estimated  mean  uses  the  data 

in  the  tall  above  and  including  the  corresponding  count. 

Counts  Frequency  Estimated 

Mean 


75 

2 

12 

3 

71 

0 

64.9 

70 

12 

69 

22 

64.9 

68 

31 

65.0 

67 

42 

65.0 

66 

34 

65.3 

65 

37 

65.5 

64 

49 

65.3 

63 

42 

65.1 

62 

39 

64.9 

61 

29 

64.8 

60 

12 

64.7 

59 

14 

64.5 

5§ 

8 

64.6 

57 

11 

64.4 

56 

14 

64.1 

Let  T1  •  63.5  and  Tg  -  62.5.  Then  n1  *  245  and  ux  - 
65.3.  The  estimated  asymptoti  **rd  deviation  of 

(n2  -  n2/ng^  is  .45.  In  this  case  n2)/n2^  -  -.55. 

Move  to  T2  and  repeat  the  process.  I*t  -  57.5  and 
T?  -  56.5.  Then  n1  -  395  and  ux  -  64.6.  The  estimated 
asymptotic  standard  deviation  of  (n2  -  ngj/n^  is  .08. 

In  this  case  (ft2  -  n^/n^  -  -.44.  Stop  at  Tx  and  use 
-  64.6  as  the  estimate. 
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TABLE  2 


Data  From  the  Scanning  Radiometer 


data  are  in  counts 

.  The  estimated  mean 

uses  the  data 

the  tail  above  and 

including  the  corresponding  count. 

Counts 

Frequency 

Estimated 

Mean 

C 

fl 

73 

2 

72 

3 

71 

8 

70 

12 

64.9 

69 

22 

64.9 

68 

31 

65.0 

67 

42 

65.0 

66 

34 

65.3 

65 

37 

65.5 

64 

49 

65.3 

63 

42 

65.1 

62 

39 

64.  Q 

61 

29 

64.6 

60 

12 

64.7 

59 

14 

64.5 

58 

8 

64.6 

57 

11 

64.4 

56 

14 

64.1 

Let  =  63.5  and 

T2  -  62.5.  Then  n1 

■  245  and  « 

65.3.  The  estimated  asymptotic  standard  deviation  of 
(n2  -  n2/n2^  is  .45.  In  this  case  (n2  -  r^J/n^  -  -.53. 
Move  to  T2  and  repeat  the  process.  Let  -  57.5  and 
T2  «  56.5.  Then  n^  «  393  and  ■  64.6.  The  estimated 
asymptotic  standard  deviation  of  (ng  -  n^/nj^  is  .08. 
In  this  case  (ft2  -  rigj/n^  ■  -.44.  Stop  at  and  use 
■  64.6  as  the  estimate. 
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6.  CONCLUSIONS 

The  technique  presented  in  this  paper  gives  a  simple 
objective  method  for  finding  a  truncation  point  for  a  dis¬ 
tribution  which  is  contaminated  over  part  of  its  range. 

For  large  samples  with  the  single  parameter  normal  it  seems 
to  work  very  well. 
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ABSTRACT 


Wr.en  estimating  atmospheric  and  surface  parameters  from  satel¬ 
lite  •’■‘’asurements  of  infrared  radiance,  one  of  the  most  diffi¬ 
cult  problems  is  that  of  cloud  effects.  It  has  been  shown  thn 
the  t'r  '  ■  ry  of  truncated  probability  distributions  can.  be  used 


to  estimate  clear  radiances  in  certain  situations.  In  this 


paper  it  is  shown  that  the  same  techniques  can  be  used  to  ob¬ 
tain  bounds  on  certain  cloud  parameters.  These  parameters  in¬ 


clude  eloud-top  temperature  and  cloud  amount.  If  the  atmes- 
r*  or’  -'mperature  profile  is  assumed  Known,  then  bounds  on 
clou.  .  ight  can  be  obtained.  These  bounds  can  then  be  used 
to  improve  results  for  certain  temperature  retrieval  technique 


Fart  of  this  paper  was  written  while  the  author  was  sunporto 
by  the  Office  of  Naval  Research  Contract  NOOOl^^TC-Ot'D 4 . 


David  S  Crosby 


INTRODUCTION 


The  primary  difficulty  associated  with  remote  sensing  of  atmo¬ 
spheric  or  surface  parameters  using  satellite  measured  infrared 
radiation  Is  the  presence  of  clouds.  In  a  previous  paper,  a 
technique  which  uses  the  theory  of  the  truncated  normal  distri¬ 
bution  to  estimate  cloud-free  radiances  has  beer,  developed.  See 
Crosby  [107-  .  In  this  paper  the  technique  of  using  the  trun¬ 

cated  nornr.  distribution  is  used  to  obtain  bounds  on  certain 
cloud  parameters.  Those  parameters  include  cloud-top  tempera¬ 
ture  and  cloud  amount.  If  the  atmospheric  temperature  profile 
is  assumed  known,  bounds  or.  cloud  height  can  also  be  obtained. 

The  technique  uses  a  large  number  of  small  fields-of-view 
(f-o-v's)  and  assumes  that  the  noise  of  the  instrument  has 
approximately  a  normal  distribution.  It  is  also  assumed  that 
atmospheric  temperature  is  a  decreasing  function  of  height.  If 
these  assumptions  are  satisfied,  it  is  then  possible  to  obtain, 
ar.  upper  bound  on  cloud-top  temperature  and  upper  and  lower 
bounds  on  cloud  amount  for  the  highest  layer  of  clouds  in  a 
composite  f-o-v. 


TilK  MODEL 

There  are  assumed  to  be  n  small  f-o-v's  in  the  composite 
f-o-v.  Let  R^,  R^,  ***,  RJ1  bo  the  measured  radiances  from  these 
small  f-o-v's.  If  f-o-v  i  has  more  of  the  higher  level  cloud 


Estimating  Cloud  Parameters  Using  Truncated  Distributions 
than  f-o-v-  ,1  and  if  the  effect  of  noise  is  reflected  then 
R.  <  R..  That  is,  a  f-o-v  which  is  covered  by  more  of  the 
higher  level  cloud  is  assumed  to  have  a  lower  true  radiance. 
If  several  f-o-v's  are  totally  cloud  covered  then  the  differs 
between  their  measured  radiance  values  is  assumed  to  be  due  t 
measurement  error  which  is  assumed  to  have  approximately  a 
normal  distribution. 

Let  h(r)  oe  the  theoretical  histogram  or  density  from  an  arm 
of  radiances  which  meet  the  above  conditions.  Then  the  lower 
part  of  the  histogram  will  have  the  shape  of  a  normal  curve. 
That  is,  there  is  a  K  such  that  if  r  <  K 

h(r)  ~  - - —  exp(-(r-u)Vf#'  )  ( 

t  r 

In  equation  (1)  is  the  standard  deviation  of  the  instrumen 
noise  and  u  is  the  true  radiance  emitted  from  a  cloud  covered 
f-o-v.  It  is  assumed  in  the  discussion  that  follows  that  4* 
•tnown,  but  this  is  not  a  necessary  assumption.  It  will  also 
assumed  that  K  >  p.  Given  that  the  data  satisfies  these  cond 
tions  the  techniques  for  obtaining  an  estimate  of  p  are  well 
developed  and  have  been  successfully  applied  to  the  problem  o 
estimating  sea-surface  temperature.  See,  for  example,  Crosby 
[197:]-  For  a  discussion  of  the  truncated  normal  see  Johnson 
and  ICotz  (1970], 
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reasonable  number  of  classes  to  use  as  the  minimum  should  cover 
approximately  two  standard  deviations.  For  example,  if  the 
standard  deviation  is  three  units  and  each  class  is  one  unit  in 


length,  then  use  a  minimum  of  six  classes. 


Form  the  adjusted 


mean 

of  the 

data  above  and 

including  class  ,i, 

/ 

■*r 

-  'T 

j  '  i=i 

■‘i 

• 

(-) 

Then 

find  V, 

from 

Table  f . 

/ 

TABL 

s:  v  vs  ~ 

.  .3  <  Z 

<  i 

• 

.00 

.01 

.02 

.03 

.04  .05 

.06 

.07 

.08 

.09 

2.77 

2.65 

2.53 

2.42 

2.31  2.21 

2.12 

2.03 

1.94 

1.86 

1.778 

1.702 

1.629 

1.559 

1.491  1.426 

1.363 

1.302 

1.143 

1.167 

1.132 

1.078 

1.026 

.976 

.927  .880 

.834 

.789 

.745 

.702 

.661 

.620 

.581 

.542 

.504  .467 

.431 

.396 

.361 

.327 

.294 

.262 

.230 

.198 

.166  .138 

.108 

.079 

.050 

.022 

-.006 

-.033 

-.060 

-.086 

-.112  -.138 

-.163 

-.188 

-.212 

-  .236 

-.260 

-.283 

-.307 

-.329 

-.352  -.374 

-.396 

-.418 

-.439 

-.460 

.0 

1  ^  2  <  4 

.1  .2 

.3 

.4 

.5 

.6 

.7 

.8 

.9 

-.481 

-.678 

-.856 

-1.020 

-1.172 

-1.314 

-1.449 

-1.578 

-1.702 

-1.621 

-1.937 

-2.050 

2.161 

-2.269 

-2.376 

-2.481 

-2.586 

-2.689 

-2.792 

-2.694 

-2.995 

-3.097 

-3.198 

-3.298 

-3.399 

-3.499 

-3.599 

-3.700 

-3.800 

-3.900 

If  V, 

c* 

>  0,  repeat, 

us  in(t 

class  ,H1 

.  Continue  until  V 

m 

<  o. 
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Then  the  estimate  for  the  counts  from  a  totally  cloud  covered 
f-o-v  is  given  by 

c  =  ^  -V  0~  . 

m  tn 

This  C  can  be  converted  to  radiance  or  to  temperature  by  the 
calibration  curve  of  the  instrument.  Given  the  assumption  that 
temperature  decreases  with  height  this  number  will  be  an  upper 
bound  for  the  cloud-top  temperature.  If  the  temperature  profile 
is  assumed  known  as  a  function  of  height  then  it  is  obvious  how 
to  use  this  temperature  to  obtain  a  bound  on  cloud  height. 


To  estimate  the  number  of  f-o-v's  which  are  totally  covered  by 
clouds  the  following  procedure  is  used. 

Let 

m 

N(km)  =  Z  f.  (4) 

m  i=l  1 


where  m  is  the  number  of  classes  used  to  estimate  C  in  Eo.  J. 
N(k  )  is  the  number  of  measurements  above  k^ .  Then  an  estimate 
for  the  number  of  f-o-v* s  which  are  totally  covered  by  clouds 


is  given  bv 


W  = 


exn(  (x-c)“~  ) 


(•:) 


and  N  =  [a],  where  [W]  is  the  smallest  integer  greater  than  or 
equal  to  W.  To  obtain  a  lower  bound  on  the  proportion  of  the 


composite  f-o-v  covered  by  cloud,  set 

PL  «  (N/M) 
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where  M  is  the  number  of  small  f-o-v's  in  the  composite  f-o-v. 

To  obtain  an  upper  bound  on  the  proportion  covered  by  the  higher 
level  clouds  the  same  technique  i6  applied  to  the  tail  of  data 
associated  with  the  higher  radiances  or  lower  counts. 

EXAMPLES 


Fif  .  1.  Histogram  of  count  data  from  Scanning  Radiometer. 
The  procedure  has  been  applied  to  data  from  the  10.5  *  12.5  u^ 
Infrared  channel  of  the  Scanning  Radiometer.  The  technique  was 
applied  to  data  blocks  of  1024  measurements.  Pigs.  1  and  2  are 
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histograms  of  such  data.  The  examples  are  from  the  Pacific 
Ocean  region  for  July  12,  1975. 

In  each  figure  K  is  the  truncation  point  used  (K  -  k^) ,  T  is  the 
temperature  estimated  for  a  completely  cloud  covered  f-o-v,  the 
assumed  standard  deviation  of  the  instrument  was  three  counts, 
the  heavy  black  curve  is  the  theoretical  density  (normal  distri- 
cution)  of  completely  cloud  covered  f-o-v's.  For  example,  in 
Fig.  1,  the  truncation  point  was  l8o  counts,  the  temperature 
was  227. K,  at  least  35  per  cent  of  the  f-o-v's  were  covered 
by  the  clouds  and  at  least  11  per  cent  were  cloud  free  (at  least 
of  the  higher  level  clouds). 


Fig.  2.  Histogram  of  count  data  from  Scanning  Radiometer. 
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In  Fig;.  .  three  different  truncation  points  or  K’s  were  used 
Note  the  small  chance  in  the  temperature  valuer.. 

CONCLUSIONS 


A  technique  has  been,  presented  which  gives  simple  and  olye  "t'vo 
estimates  of  bounds  on  certain  cloud  parameters.  In  many  situ¬ 
ations  the  technique  may  net  give  results  which  will  be  indepen¬ 
dently  useful.  However,  they  can  be  used  as  starting  points  cr 
checks  on  situations  of  the  radiative  transfer  equation.  The 
possible  value  o°  the  technique  comes  from  the  following  con¬ 
siderations:  with  the  cloud  data  given  by  this  technique  cer¬ 

tain  techniques  for  retrieving  atmospheric  parameters  which  hav< 
not  been  practical  in  t tie  past  may  now  be  practical;  since  the 
technique  is  so  simple  it  could  bo  Implemented  on  the  satellite, 
and  hence  would  solve  many  of  the  data  *’low  problems  associated 
with  small  f-o-v  instruments. 
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